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Abstract. We discuss permutation representations which are obtained by 
the natural action of S n x S n on some special sets of invertible matrices, de- 
fined by simple combinatorial attributes. We decompose these representations 
into irreducibles. The multiplicities involved have a nice combinatorial inter- 
pretation. We also generalize known results on asymptotic behavior of the 
conjugacy representation of S n - 

1. Introduction 

The (0,l)-matrices have a wide variety of applications in combinatorics as well as 
in computer science. A lot of research had been devoted to this area. By considering 
the set of n x n (0,l)-matrices as a boolean monoid and relating them to posets, 
one can get interesting representations of S n . Note that S n x S n acts on matrices 
by permuting rows and columns. Some aspects of the corresponding equivalence 
relation are treated in [J] and [0]. A simultaneous lexicographic ordering of the 
rows and the columns using this action is shown in |MM| . 

The above action of S n x S n gives rise to a permutation representation of S n x S n 
on (0,l)-matrices. If we diagonally embed S n in S n x S n we get a generalization of 
the conjugacy representation of S n - 

Adin and Frumkin |AF| showed that the conjugacy character of the symmetric 
group is close, in some sense, to the regular character of S n . More precisely, the 
quotient of the norms of the regular character and the conjugacy character as well 
as the cosine of the angle between them tend to 1 when n tends to infinity. This 
implies that these representations have essentially the same decompositions. 

Roichman [J^ further points out a wide family of irreducible representations of 
S n whose multiplicity in the conjugacy representation is asymptotically equal to 
their dimension, i.e. their multiplicity in the regular representation. 

In this paper we use the action of S n x S n on the (0,1)- matrices to define two 
families of representations on a family of orbits of this action. The first family forms 
an interpolation between the regular representation of S n x S n and the 'diagonal 
sum' of the irreducible representations of S n : (§)\^ n S x ® S x . The other family is 
a generalization of the conjugacy representation of S n . In both cases we calculate 
characters and present the decomposition of these representations into irreducibles. 
The second family of representations can be seen as an extension of the results of 
|AF] and 0. 

The rest of this paper is organized as follows: In Section|5]we give some prelimi- 
naries on permutation groups and their conjugacy classes and on representations of 
S n - In Sections 131 and ITTTTI we define the actions of S n x S n and of S n on invertible 
matrices. In Sections 01 and El we specialize them into the case of (0,l)-matrices. 
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We also discuss the two permutation representation they give rise to and calculate 
their characters. In SectionElwe show some asymptotic results concerning the gen- 
eralization of the conjugacy representation of S n . In Sectional we decompose the 
representation of S n x S n mentioned above into irreducible representations. In Sec- 
tion[S]we consider the actions described above on color permutation groups instead 
of (0,l)-matrices. 



2.1. Permutation Groups. S n is the group of all bijections from the set 
to itself. Every tt 6 S n may be written in disjoint cycle form usually omit- 
ting the 1-cycles of tt. For example, tt = 365492187 may also be written as 
tt = (9, 7, 1, 3, 5)(2, 6). Given tt, t 6 S n let ttt := tt o t (composition of func- 
tions) so that, for example, (1,2)(2,3) = (1,2,3). Note that two permutations are 
conjugate in S n if and only if they have the same cycle structure. In this paper we 
write tt ~ a if the permutations tt and a are conjugate in S n . We denote by S n 
the set of conjugacy classes of S n and by CV < S n the centralizer subgroup of the 
element tt £ S n . Let C(tt) C S n denote the conjugacy class of the element tt e S n . 
By supp(tt) we mean the set of digits which are not fixed by tt. An element tt 6 S n 
with \supp(tt)\ = t can be considered as an element of St and then C* denotes the 
centralizer subgroup of the element tt in S t while C"(7r) denotes the conjugacy class 
of the element tt in St- TTkTT n -k denotes an element of Sk X S n -k where tt^ 6 Sk 
and TT n -k € S n -k- 

C kx(n-k)^ klTn ^ denotes the 

conjugacy class of the element TTkitn—k hi Sk x S n ~k- 

There is an obvious embedding of S n in GL n (¥) where is F is any field. Just 
think about a permutation tt € 5,1 as an n x n matrix obtained from the identity 
matrix by permutations of the rows. More explicitly: for every permutation tt € S n 
we identify tt with the matrix: 



2.2. Color permutation groups. For later use, we define here the color permu- 
tation groups. For r, n € N, let G r ,n denote the group of all n by n monomial 
matrices whose non-zero entries are complex r-th roots of unity. This group can 
also be described as the wreath product C' r I S n which is the semi-direct product 
C™ x S n , where C™ is taken as the subgroup of all diagonal matrices in G r . n . For 
r = 1, G r ,« is just S n while for r = 2, G r ,„ = i?„, the Weyl group of type B. 

2.3. Representations. 

2.3.1. Permutation representations. In this work we deal mainly with permutation 
representations. Given an action of a group G on a set M, the appropriate rep- 
resentation space is the space spanned by the elements of M on which G acts by 
linear extension. We list two well known facts about permutation representations. 

Fact 2.1. The character of the permutation representation calculated at some g G G 
equals to the number of fixed points under g. 

Fact 2.2. The multiplicity of the trivial representation in a given permutation 
representation is equal to the number of orbits under the corresponding action. 



2. Preliminaries 
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An important example we will use extensively in this work is the conjugacy 
representation which is the permutation representation obtained by the action of 
the group on itself by conjugation. 

2.3.2. Representations of S n . Let n be a nonnegative integer. A partition of n 
is an infinite sequence of nonnegative integers with finitely many nonzero terms 

oo 

A = (AijAa, . . .), where Ai > A2 > ••■ and ^« = n - 

i=i 

The sum ^2 Ai = n is called the size of A, denoted |A|; write also A h n. The 
number of parts of A, ^(A), is the maximal j for which Xj > 0. The unique partition 
of n = is the empty partition = (0,0,...), which has length t (0) := 0. For a 
partition A = (Ai, . . . , A&, . . .) define the conjugate partition A' = (A' l5 . . . , A£, . . .) 
by letting A- be the number of parts of A that are > i (Vi > 1). 

A partition A = (Ai, Afc) may be viewed as the subset 

{(^m^^M-^Ajcz 2 , 

the corresponding Young diagram. Using this interpretation we may speak of the 

intersection A n p, the set difference A \ p and the symmetric set difference XAp of 

00 

any two partitions. Note that |AA/i| = \Xk — fik\- 

fe=i 

It is well known that the irreducible representations of S n are indexed by par- 
titions of n (See for example [5a]) and the representations of S n x S n are indexed 
by pairs of partitions (X, p) where A, fi h n. For every two representations of S n , 
A and p, we denote by m(A, p) the multiplicity of A in p. If we denote by ( , ) the 
standard scalar product of characters of a finite group G i.e. 

(Xi,X2) = j^r XI Xi(7r)X2(7r) 

then m(A,p) = (xx,X P )- 

Similarly, m((A,/i), tp) denotes the multiplicity of the representation of S n x S n 
corresponding to the pair of partitions (A, p) , A h n , p h n in the decomposition 
of if, where if is any representation of S n x S n - 

We cite here for later use the branching rule for the representations of S n . We 
start with a definition needed to state the branching rule. 

Definition 2.3. Let A h n be a Young diagram. Then a corner of X is a cell 
G A such whose removal leaves leaves the Young diagram of a partition. Any 
partition obtained by such a removal is denoted by A~ . 

Proposition 2.4. Sa // Ah n then 

A- 

3. The action of S n x S n on invertible matrices 

Definition 3.1. Let G be a subgroup of S n x S n and letF be any field. We define 
an action of G on the group GL n (¥) by 

{■k, a) • A = nAa^ 1 where (ir, a) € G and A e GL„(¥) (I) 
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It is easy to see that this really defines a group action. 

In this work we deal only with the cases: G — S n x S n and G = (Sk X S n —k) x 
(Sk x S n -k)- 

Definition 3.2. Let M be a finite subset of GL n (¥), invariant under the action 
of S n x S n defined above. We denote by olm the 'permutation representation of G 
obtained from the action (1) . In the sequel we identify the action (1) with the 
permutation representation olm associated with it. 

3.1. A generalization of the conjugacy representation of S n . In this section 
we present a conjugacy representation of S n on a subset M of GL n (¥). 

Definition 3.3. Denote by (3 the permutation representation of S n obtained by the 
following action on M . 

tt o A = (tt, 7t) • A = ttAtt^ 1 (2) 
The connection between % and /3 m is given by the following easily seen claim: 
Claim 3.4. Consider the diagonal embedding of S n into S n x S n . Then 

Pm = a M if™ xS " ■ n 

Theorem 3.5. For every finite set M C GL n (¥) invariant under the action (1) of 

S n x S n defined above: 

If 7T and a are conjugate in S n then 

XaM ((tt, a)) = Xa M ((tt, tt)) = X fi M W = #{A € M I ttA = Att} . 

// 7r is not conjugate to a in S n then 

Xcxm {{^,0-)) =0. 

Proof. See Theorem 4.5 in |US1- □ 

Example 3.6. Take M = S n (embedded in GL n (¥) as permutation matrices). In 
this case @m is just the conjugacy representation of S n and a direct calculation 
shows that for every tt £ S n : 

X/3„M = \C n \= Xa M (ir,ir). 
For every irreducible representation of S n corresponding to a partition X one has: 



m(A, Pm) = y^XA(C) where C runs over all conjugacy classes of S n . 
c 

Moreover, 

m((X,X),a M ) = 1, 

and 

m((A, /i), cum) = when A ^ [i. 

This means that aM — ©Ahn ® where S x is the irreducible S n - module 
corresponding to X. This fact will be implied by substitution k = in Proposition 
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4. The action of S n x S n on (0,1)-matrices 

In this section we specialize the action (1) of S n x S n defined in Sectional to 
(0,l)-matrices. Consider the group G = GL n {T^i). For every A £ G denote by o(A) 
the number of nonzero entries in A. One can associate with A a pair of partitions 
of o(A) with n parts (77(A), 9(A)) where 77(A) describes the distribution of nonzero 
entries in the rows of A and 0(A) describes the same distribution for columns. For 
example, if: 



.4 



1 


V 1 



then 77(A) = (4, 3, 1,1)1-9 and 9(A) 




1 
1 
1 

(3,3,2,1 






V 

) h 9. 



If we fix a pair of partitions (77, 9) then the set of matrices corresponding to (77, 9) 
is closed under the action (1), but this action is not necessarily transitive on such 
a set, i.e. it can be decomposed into a union of several orbits. For example, if 



A = 















f 1 


1 





°\ 





1 


1 





and B = 


1 





1 








1 











1 








V 








°) 




\P 








V 



then r/(A) = 9(A) = 77(B) = 9(B) 
A 7^ nBa for any 7r, a S 64. 

We present now a family of subsets of GL n (Z2 
be orbits of our action: 



2,2,1,1) but it can be easily shown that 
which will be proven shortly to 



Definition 4.1. 

H° = {A eG\r,(A) = 9(A) = (1,1,1,..., I) = 1™} 
Hi = {AeG\ 77(A) = (n, 1,1,..., 1), 9(A) = (2, 2, . . . , 2, 1) = 2"- 1 1} 
H% = {AeG\ 77(A) = (n, n - 1, 1, ... , 1), 9(A) = (3, 3, . . . , 3, 2, 1) = T~ 2 2 1} 



H* = {AeG\ V (A) = (n,n-l,...,n-(k-l),l,...,l), 

9(A) = (k+l,k+l,...,k+l,k,k-l,...,2,l) = (k + l) n - k k (k- 1) ... 21} 



HZ = {Ae\ 77(A) = 9(A) = (n, n - 1, n - 2, . . . , n - (k - 1), . . . , 3, 2, 1)} 

Note that in the above example A S H%. 
A few remarks on the sets H k are in order: First, note that \H^\ = 77,1(71,)^. Sec- 
ondly, note that is S n , embedded as permutation matrices. Also note that the 
set U H\ is closed under matrix multiplication and matrix inversion and is actu- 
ally isomorphic to the group SVt+i- Another simple observation is that i/™ = H^~ x . 

In order to prove that the sets H k are transitive under the action we need the 
following definition: 
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Definition 4.2. Denote by U n ,k the following binary n x n matrix : the upper left 
k x k block is upper triangular with the upper triangle filled by ones, the upper right 
kx (n — k) block is filled by ones, the lower left (n— k) x k block is the zero matrix 
and the lower right (n — k) x (n — k) block is the identity matrix I n -k- 

Proposition 4.3. Each set H% is transitive under the action (1). More explicitly, 
Hi = {nUn^a | 7T, <7 £ S n } 

Proof. We will prove that U n ,k is a representative of H%- 

Take an arbitrary matrix A £ H 1 ^. By the definition of A has a unique 
row which is filled by ones. Choose a permutation matrix tti such that this row is 
the first row of -K\A. By the definition of H%, tt\A has a unique column of type 
(1,0,..., 0) T , thus one can choose a permutation matrix o\ such that this is the 
first column of tt\Ao\. Now consider the (n — 1) x (n — 1) sub matrix A\ obtained 
from ~K\Ao \ by deleting the first row and the first column. It is easy to see that 
A\ e -ff^Zj and we can repeat the process described above. Note that the first row 
and the first column of A will not be changed since their entries numbered 2..n are 
identical, (only l's in the first row, only 0's in the first column. Continuing this 
way we get 

7T t 7Ti_l • • • TTlAai ■ ■ ■ Ot-\0- t — U nt k 

which proves that H% C {-KU n ^o \ ir, a € S n }. The other inclusion, i.e. {irU n ^o- \ ir, a e 
S n } C H^, follows directly from the definitions of U n .k and □ 



In particular 



/I 

1 

••• 

\0 ••• 



••• 0\ 

... o 

1 

1/ 



a | 7T, a e S n \ = S n 



(I 






1\ 





1 
1/ 



a 1 7T, cr e S„ 



n 







1 \ 

1 
1 





a | 7T,CT E S n 









1 / 
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/ll 1 1 ••■ 1\ 

1 1 1 ••■ 1 
1 1 ••■ 1 

Hn = { K ■ . . . . ■ cr I 7T, CT G S„ 

o 1 1 

\o 1/ 

For the case k = n the permutation representation a H k can be easily described: 

Proposition 4.4. TTie representation a^n is isomorphic to the regular represen- 
tation of S n x S n . 

Proof. As was shown in the proof of Proposition 14.31 if™ = {irU n ^ n a | 7r, a G S n }, 
where is the upper triangular matrix with all the upper triangle filled by 

ones. Define a bijection tp : H™ — > S„ x S n by p(irU ntn a) = (ir, cr _1 ). Since 
each row (column) of U n _ n has a different number of 1-s (from 1 to n), we have: 
TTiU njn o~i = 'KiU n ^ n 02 if and only if m — it2 and o\ = 02 • This means that the 
mapping tp is well defined and bijective. Now: 

(lu, t) • (TtU n>n a) = WTrJ/n.nCrr -1 i— > (w7r, t<t _1 ) = (w, t)(tt, c -1 ). 

Thus iy9 is an isomorphism of S n xS n - modules between if™ and the (left) regular 
representation of S n x S n . 

□ 

4.1. A natural mapping from iJ„ onto S n . In this section we present a surjec- 
tion between the representation of S n x S n on if *; to the representation of S n x S n 
on Sn. We will use this mapping later when we decompose the permutation repre- 
sentation a into irreducibles representations. 
We deal first with H%. 

Definition 4.5. Define the mapping t : H™ — ► S n by 

{llU n . n a) >-> 7T(7. 

This mapping is well defined since by the definition of U n , n 

nU n>n a = pU n . n T n = p and a = r. 

It is easy to see that this mapping is a surjection and, moreover, | i 1 (tt) | = n\ 
for any 7r G S n . (Indeed, any fixed it £ S n can be represented in exactly n! ways in 
the form 7r = xy since choosing x G S n in n! ways we must take y — x _1 7r). 

Proposition 4.6. The mapping t preserves the action a of S n x S n on H™, i.e. 

t(wAa) = wt(A)cr for any Ae H™. 



Proof. Any A G H™ can be represented A = pU n ,nT for some p, r G S^. Thus by 
the definition of t we have 

t(irAa) — t(jrpU n . n Tcr) = Ttpra = Trt(pU n ^ n T)o~ — irt(A)a. 

□ 

Now, in a similar way, for every n, 1 < k < n we define to T n ^ : — ► S n as 
follows: 
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Definition 4.7. Define the mapping T„ t k ■ — > S n by T n ^k{nU n .kO') = ^a. 

Notice that unlike the case of the sets i?™, here we have to show that the mapping 
T n .k is well defined since a matrix A G H% can be written in the form irU n ,kO~ in 
more than one way. Indeed, T n ^ is well defined since if n\] n ^a — U n ^k then as the 
first k rows (columns) of U n ,k are different, we have n, a G {e} x S n —k- Thus we 
can write ir = Tt n -k and a = a n -k and it is easy to see that one must have: ttu = e 
which says that T n ^(irU n , k cr) = e = T n ,h{U n ,k)- 

Just as was the case in if" we have that: 

Proposition 4.8. The mapping T n< k preserves the action a of S n x S n on H 1 ^, i.e. 

T n>k {-KAa) = TvT„, k (A)a for any A G H%. □ 

It is also clear from the definition that T n k is onto and it is easy to see that 
I^W|=fc!(«) = (n) fe . 

5. The representation f3 M for M = H%. 

In it was proven that the conjugacy representation of S n contains every 
irreducible representation of S n as a constituent. The representation (3 defined in 
Section ETH is a type of a conjugacy representation of S n on H*. 

Proposition 5.1. Denote the conjugacy representation of S n by tp. Then every 
irreducible representation of S n is a constituent in (3 H k . In other words 

m (A, > for any A h n. 

where m(A, f3 H k) denotes the multiplicity of the irreducible representation corre- 
sponding to A in (3 H k . 

Proof. Denote the conjugacy representation of S n by ip. In [F] it is shown that 
m(X,ip) > for any A h n. By proposition ^. 81 T n ^ commutes with the conjugacy 
representation of S n and thus it gives rise to an epimorphism from [3 H k onto the 
conjugacy representation of S n . Hence, by Schur Lemma, 

m (A, (3 H k) > m (A, t/>) > 0. 

□ 

We turn now to the calculation of the character of f3 H k . By the definition, we 
have: 

X0 H k M (= Xa H u t)) = #{A G U k n I ttA = An} 
but we can achieve much more than that: 

Proposition 5.2. 

X/3 ffS ( 7r ) = |Cir|(ra- \supp(ir)\) k = (n- \supp(Tr)\) k xconj(ir) 
where xconj is the conjugacy character of S n . 

Proof. Denote t — \supp(ir)\. We can assume that tt is of the following form: 



7T = 

where 7r t G St without fixed points. 



TTt 
In-t 
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We have to calculate the number of matrices A £ H% which commute with n. 
We recall the definition of H^: 

H* = {A£G\ n(A) = (n, n - 1, . . . ,n - (k - 1), 1, . . . , 1), 
9(A) = (k + l,k + l,...,k + l,k,k-l,...,2,l) = (fc + l)"~ fe k(k — l) ... 21} 

For every A £ H% denote by 5(A) the vector of the row sums of A and by e(A) 
the vector of the column sums of A. 
For example: 
If 



A = 



1 










1 


1 


1 











1 





V 


1 


1 


V 



then 5(A) = (1, 3, 1,4) 1= 9 and e(A) = (3, 2, 3, 1) N 9. 

Note that every A £ has k rows with row sums ranging from n to n — k + 1 , 
these will be called 'long rows'. The other n — k rows are monomial. Likely, A 
has k columns with column sums ranging from 1 to k, these will be called 'short 
columns'. The other n — k columns have k + 1 1-s each. 

In it A only the first t rows of A are permuted while in Air only the first t columns 
of A are permuted. 

Note also that for every n £ S n we have: 5(Att) = 5(A) and e(nA) = s(A). 

Since all of the 'long' rows of A have different row sums, if one of the first t 
rows of A is 'long' then 5(nA) ^ 5(A) = 5 (An) and thus An ^ nA. Hence we can 
assume that all the 'long' rows in A are located after the first t rows of A. This 
implies that the first t rows of A are monomial. By similar arguments, the 'short' 
columns are located after the first t columns. 

We claim now that the upper right tx (n — t) block of A is the zero matrix. 
(Indeed, if Aij = 1 for some 1 < i < t and t + 1 < j < n then for each 1 < i' < t 
with i' ^ i we have: Ayj ^ 1 since A is invertible. Now, in nA this 1 moves to 
another place while in An it is left in its original position). 

Note that we have now that the upper left t x t block of A is a permutation 
matrix which commutes with 7r t in St- 

We turn now to the lower left (n — t)xt block of A. This block has k 'long' rows 
and n — t — k monomial rows. Note that by the consideration we described above, 
the monomial rows contribute only -s to this block. On the other hand, since the 
'short' columns are located after the first t columns, for every 1 < j < t the column 
numbered j has exactly k + 1 1-s. But for each j one of this 1-s must be located at 
the upper left txt block since this block is a permutation matrix, hence the 'long' 
rows contribute only 1-s to the lower left (n — t)xt block. 

Finally, as can be easily seen, the lower right (n — t)x(n — t) block is an arbitrary 
matrix of H^_ t . 

Now calculate: 

X^) = #{A£H k n \nA = An} = 

= \Ci\\H k n _ t \ = \C t J(n-t)\(n-t) k = 

= \C v \(n - t) k = (n- \supp(n)\) k xconj(n) 

□ 
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Note that Xa k i^, n) = XP Hk M = when t — \supp(ir)\ > n — k. On the other 
hand, if n - t > k then Xa g H ("", i") = Xp H u 00 ^ 0. 

We turn now to the calculation of the multiplicity of every irreducible represen- 
tation of S n in f5 H k . 

Proposition 5.3. Let A h n. 

m(A,/3^)= ^ X A(C)(n~| suw (C)|) fc 

where S n denotes the set of conjugacy classes of S n . 
Proof. As above t = \supp(n)\. Then 



i(X,f3 H k) = (xx,Xff H& ) = |^-| E X^XPukW = 



tt£S„ 



= A E X\{«)\C*\(n - t) = £ XA (^)-^_(n - t ) A = 

= 5„ XA(7r) i^)i (n " %= 

= E |C|XA(C)p(nH^MCOI)fe= E XA(C)(n-|,s UW (C)|)A 

ces„ ! 1 ces„ 



□ 



6. Asymptotic behavior of the representation @ h h. 

In this section we generalize the results of Roichman , Adin, and Frumkin |AF| 
concerning the asymptotic behavior of the conjugacy representation of S n . These 
two results imply that the conjugacy representation and the regular representation 
of S n have essentially the same decomposition. In our case, as we prove in this 
section, the representation (3 H k is essentially (n)u times the regular representation 
of S n - We start by citing the result from 

Theorem Rl Let m(A) be the multiplicity of the irreducible representation S x 
in the conjugacy representation of S n , and let / A be the multiplicity of S x in the 
regular representation of S n . Then for any < e < 1 there exist < 5(e) and N(e) 
such that, for any partition A of n > N(e) with max{^ , ^} < 5(e), 

l-e< — ^ < 1 + e. 
f x 

The following generalization of this theorem is straightforward: 

Proposition 6.1. For any < e < 1 there exist < 5(e) and N(e) such that, for 
any partition A of n > N(e) with max{^- , — } < 5(e), and for any k < n 
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Proof. In [R] it is shown that for any < e < 1 there exist < 5(e) and N(e) such 
that, for any partition A of n > N(e) with max{^- , ^} < 8(e), 

MA) - / A | - I E Xx(C) f x \ = | £ Xx(C)\ < ef\ 

which immediately implies Theorem Rl. In our case we have the trivial observation 
(n — \supp(C)\)k < (n)k which together with the above gives us 

\m(\,p Hkn )-(n) k f X \ = \ XJ Xx(C)(n-\supp(C)\) k -(n) k f x \ = 

ces n 

= I E Xx(C)(n-\supp(C)\) k \ < (n) k \ ]T Xx(C)\ < (n) k ef x 

C^id C=£id 

and our claim is proved. □ 

The following asymptotic result from [AF] can also be generalized for the char- 
acters Xl3 H s • 

Theorem AF Let x'r an d X'clnj ^ c tnc regular and the conjugacy characters of 
S n respectively. Then 



hm »*y» =1, 

™ \\x ( cU\ 



Ufi iXConj/ 



™llxk n) Mlx^ 



where || * || denotes the norm with respect to the standard scalar product of char- 
acters. 

Our generalization looks as follows: 
Proposition 6.2. In the notations of Theorem AF 

i im ll(")*xk B) ll =1 

n ^°° \\X0 H J\ 

{{n)kX ( R \xp Hk ) (Xr\x0 h h) 
lim - = lim — ^ - = 1 , 

"^HWrfll-llx^JI ™ llx^ll ■ \\xe B J 

where k is bounded or tends to infinity remaining less than n. 
Proof. Denote for every < k < n 

p(q , \ - {(n - \supp(C)\) k ) 2 

tk(s n )= 2^ jq • 

ces n 
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Now 

2 



= h E (x^w) 2 = A E i^i 2 ((^~ i«*ppWD*) 3 = 

= ^ E |^yp (( n - I su ppWIW 2 = 

, V"^ ((»- I^Pp(TT)l)fc) 2 = 



Z_, l C l [q2 = nLF fc (£„) 

ces„ 



It is well known that for every character \ of S n : 

(X { n\x) = x(e) and || x £>|| 2 = (x^,^) = «!, 

hence 

\\Xfi B J 2 = F k (S n )\\xP\\ 2 . 

Moreover, we have: 



(XR,Xf} Hh ) Xf3 Hk (e) 



_ (n) k n\ _ (n) k 



VrJ.y/n\F k (S n ) y/FtfSJ 
It is obvious from the definition of F k (S n ) that: 



(H fe ) 2 < F k (S n ) < ((n) k ) 2 J2 

ces„ 



In [AF] it is proved that 



lim — 



" >- — * | c I 



ces„ 

which establishes our claim. □ 



7. The representations au for Af = i?* 

In this section we deal with the representations a^k defined in Section [21 We 
use the branching rule and the Frobenious reciprocity to decompose these repre- 
sentations into irreducible representations of S„ x S n . As we have already seen 
in example 13.61 ajjo = @ Ahn S x ® S x while an™ is the regular representation of 
S n x S n — A phn f x f p S x ® 5 P and thus a H k can be seen as a type of an interpo- 
lation between these two representations. 

First, concerning the character of a H k, by combining Proposition 15 . 21 and Claim 
13.51 together we get: 
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Xa Hk (7r,0-) 



|C^|(n — \supp(ir)\)k , 7r and a are conjugate in S n 
, otherwise 



We turn now to the direct calculation of the multiplicity of an irreducible represen- 
tation of S n x S n in a H k . 

Proposition 7.1. For any n and any < k < n 

m((X,n),a H h) = — Z XaWXmW(i~ \supp{n)\) k 

Proof. Recall that t = \supp(tt)\. 

m((X,n),a H k) = (X(A, M ),Xa ffS )s„xs„ = 

1 " (7r,cr)GS„x5„ 

= 7^1)2 Z ^ ^ (fa = 

(7T,Cj) : 7T~CJ 



[n 



TT2 Z I^WIxaWxmWx^ (fa 71 ")) 



= 7^2 E I^WIXA(7r)x M W|CV|(n-i)* = 
E I^WIxaWx^Ctt)— ^r(n-*)fc 



-7 E Xa(7t)x m (7t)(^- |supp(7r)|) A 



n. 

ir£S n 



□ 



The boundary cases k = and k = n are discussed in Example 13.61 and Proposition 
14.41 respectively. 

We list now several simple corollaries from Proposition [7| 
Corollary 7.2. 

m ((A, n),a H k) = m ((A', fi'),a H k) , 
where A' is i/ie partition conjugate to A, and 

m((X^),a H k) = m((fi,X),a H k) . 

Proof. The first statement is due to the well known fact that xa' (f) = sgn^^xxi^)- 
The second one is obvious. □ 

Note that m (((n), (n)) , a H k) = 1 since the action a of S n x S n is transitive on 
H\. This implies the following corollary: 
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Corollary 7.3. 

(n — \supp{ir)\)k — nl for all < k < n. 

xeS„ 

□ 

Corollary 7.4. 

m ({(n), 1 "), = — ^2 sgn(n)(n-\supp(n)\) k = Q 

Proof. Clearly 

m(((n),l n ) ,a H k) = — } sgn(n)(n~ \supp(n)\) k < 

<— V (n - |supp(7r)|) fc = m (((n), (n)) , a H k) = 1 

But m (((n), 1") , a H k) is a non negative integer, therefore it must be zero. □ 

7.1. A combinatorial view of a H k. In this section we present another approach 
to the representation k . This approach will give us a combinatorial view on the 
multiplicity formulas we calculated in the last section. 

Definition 7.5. Define the following subset of H*: 

= {lTk'K n -kUn,kO~kO~n-k \ T^k^k € Sfc and TT n -k,O n -k € S n -k} ■ 

The set W k is the orbit of the matrix U n ,k under the action a restricted to the 
subgroup (S k x S n -k) x (S k x S n -k)- 

Definition 7.6. Denote by U) n> k the permutation representation of the group (Sk x 
S n -k) x (Sk x S rl -k) on Wn corresponding to the action a. 
Claim 7.7. 

where Rk is the regular representation of Sk x Sk- 

Proof. We can view the action a of (Sk x S n -k) x (S^ x S n -k) ° n as composed 
of two independent actions. One of them is the action of Sk x Sk on i?^ (this 
corresponds to the the upper left block of U n: k) and the other is an action of 
Sn-k x Sn-k on S n -k (this corresponds to the lower right block of J7„,fc). The 
permutation representation corresponding to the first action is actually the regular 
representation of Sk x Sk (see Proposition ^. 4(1 while the second one is @ phn _ fe S p ® 
S p (see Example E3J. 

□ 

This implies the following: 
Claim 7.8. 

!0 when irk ^ e or Ok e 

when ir n -k is not conjugate to er n _ fe in 

(kl) 2 \C™~ k k | when ir k = a k = e and 7r„_ fe — er n _ fe in 

□ 
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We can use u> n ,k to get information of a n ,k- 
Proposition 7.9. 

a H H = uj n>k Tfs fc x x S s„_ fc)x( s fcXS „_ fc) 

Proof. Write G = S„ x S n and H — (S k x S n -k) x (5* x S n - k ) and identify G/i7 
with a prescribed set of left transversals of H in G. We clearly have: 

= {g • U n , k \geS n x S n } = 
{(ah)»U n ,k \aeG/H,heH} = 
{a • (h • J7 n , fc ) | a e G/iJ, h e H} = 
{<r.W*\o-EG/H} = 
l+J <r«W n fe 

cr£G/H 

where td denotes disjoint union. 
This implies that 

a ^ = 17 • spancW* = w n>k f { g* x S g n _ k )x(S k xS n _ k ) 

aeG/H 

as claimed. □ 

We use now the Frobenius reciprocity to obtain the multiplicity of any irreducible 
representation of S n x S n in a H k . 

Proposition 7.10. Let < k < n and let A, \x be partitions of n. Then 
m ((A, fi) , a H k) = (xx i|*_ h ,Xp if"_ fe ) 

or in other words: 

X,fihn 

Proof. Recall that X(A,^) is the character of the irreducible representation S x ® S^ 
of S n x S n - Then, by Frobenius reciprocity, Claim 171^1 and Proposition l7.9l we have 

m ((A, fi) , a H k) = (Xa Hk , X(A,/i)> = 

= (Xui„, k tfs fcX s"„_ fc )x(S , A ! xS„_ fc )'X(A, AI )) = 
= (Xu n , k ,X(X,u) ifs fc X xs"„_ fc )x(S fc xS„_ fc )) = 

^ Xw»,* (KkKn-k, CTfeCT„-fc)xA(7rfc7rn-fc)XA'( Cr fc Cr ™-fe) = 



(fc!) 2 ((n-fc)!) 2 

= /^,j\2|'/ _ j^yfj2 ^ y X^n.fc (^n-fej 7r n-fc)XA( 7r n— fc)X/j( 7r n— fe) = 
' \C n -\^-k)\{k\) 2 \C^ k k \xx^n- k )X^n- k ) 



(fc!) 2 ((n-*)!)a ^ 



1 7 ' 7T n _ fc eS n _Jl, 



- (XA is" oXp is",,) 
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□ 

The number (x\ l s n k ,Xn if™ ) nas a very nice combinatorial interpretation. 
It follows from the branching rule that this is just the number of ways to delete k 
boundary cells from the diagrams corresponding to the partitions A and fi to get 
the same Young diagram of n — k cells. By the branching rule (see Proposition ^. 4(1 
we have thus: 

Claim 7.11. 

(xa if:_ fc ,XM ik-J =° when |AA Ai |>2fc 
and it does not vanish otherwise. 

Corollary 7.12. 

m ((A, /i) , (Xjjk) = when |A A /x| > 2fc 

and 

m ((A, /x) , o^fc) 7^ tu/ien |A A /i| < 2fc. □ 

7.2. Some asymptotic results. In this section we use the fact that j3 H k — 
a H k l s s n/Sn to obtain some asymptotic results. 

Embed S„ in S„ x S n as the diagonal subgroup. For A, v h n, denote 



= 3 X! XA(7r)x Al (7r)x !/ (7r). 



7Au^ 

We have the following: 
Claim 7.13. For A h n 

S A tsI xS "= lx^®S p 
Proof. Let jj,,v\- n. By the Frobenius reciprocity, 

m(( / x,z,),s'Mf: xS ") = (x(^)»XA Ti;; xS "> = 



(X(^) 4I^ x5 ",Xa) = -7 X] XaMXmMXj/M = 7a,» 



n! 



□ 



Remark: The numbers 7.^1/ appear in [Md, page 115] in the context of the Schur 
functions within the following formula: 



SA 



(xy) = ^2 JXiiuS^{x)s v (y), 



where x = (x\, x%, ...), y = (yi, y%, ■ ■ ■) and (xy) means the set of variables Xiyj. 
We have now the following asymptotic result: 

Proposition 7.14. For any < e < 1 there exist < 8(e) and N(e) such that, for 
any partition A of n > N(e) with max{^- , < 8(e), 

1 - e < — = < 1 + e. 
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Proof. By Proposition 16.11 we have: 



l-e< 



(A, (3 H k 



< 1 + e. 



(n)kf x 
so we have to show that 

m(\,f3 H k) = (Xn is^Xu l'sl_ k )l^- 

Indeed by the previous claims and the Frobenius reciprocity: 

m(\,f3 H k) = (Xf3 Hk ,X\) = 



(Xa Hk lf^ XS ",XA) = (Xa Hk ,X\ Ts^^") 



S„ x S n 



~ (Xn ls™_ k <>Xv l5™_ fc )7AM^- 

fj,,v\-n 



□ 



Substituting in the above proposition k = and k = n we get the following: 
Corollary 7.15. 

1 - e < — ^— ^ < 1 + e, 



1 ~ £ < - HFa < 1 

n\f x 



■ e. 



a 



Remark: The first statement of this corollary follows from Theorem Rl and 
the equality 

^7Amm = Xx(C) 
which follows from the character orthogonality relation. 

8. The actions a and [3 on colored permutations 

In this section we introduce actions of S n and S n x S n on another family of sets, 
namely the colored permutation groups. We start with the actions on B n — C2\S n . 

8.1. The action a of S n x S n on signed permutations. Consider the action a 
of S n x S n on B n . We start by describing the orbits of this action. 

Definition 8.1. For every < k < n define 

X„ = {Ae B n | A has exactly k minuses}. 



For example 

^ 1^ 
-10 
10 
\0 0-10/ 

It is easy to see that the sets X* form a partition of B n . Also, note that 
\X^\=n\(^. 



e xl 
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Claim 8.2. Each set is an orbit under the action a of S n X S n on B n , i.e. 

X n = {nUn^a | 7T, CT g S n }, 

r~r ( —Ik ®kx(n-k) 

Un,k = I n ,- V 

and J; zs the identity t x t matrix. 

Proof. Let A <E Jf*. Note that due to the semi direct decomposition B n = C% x S n 
we have A = ZP where Z is a diagonal n x n matrix with only ±l-s on the main 
diagonal and P is a permutation matrix. Assuming that the rows numbered ii, ...,ik 
in A have minuses, denoting t = (1, ii)(2, ig) • • • (k, ik) we have tAP t = U n ,k- 

'□ 

We decompose now the representations ax * into irreducible representations just 
as we did in the previous section. 

Definition 8.3. Define the following subset of X*: 

W„ = yRk'Rn-kU n ,kO'k°'n-k l^fciC/s € Sk and n n - k ,a n -k G • 

The set W% is the orbit of the matrix U n k under the action a by the group 
(S k x S n -k) x (S k x S n -k)- 

Definition 8.4. Denote Q ni k the permutation representation of the group (Sk x 
S n —k) x (Sk x S n -k) which is obtained from the action a of this group on the set 
W h . 

yv n • 

The proof of the following simple observation is similar to the proof of Claim 

o 

Claim 8.5. 



p\~k J \ p\~n — k 

i \ f \ c L\\ C n~ k t \ whemr k TT n -k ~ <7 k o- n -k in Sk x S n -k 

Xz n JWn- k ,o- k a n -k) = \ Q — QthRrwise 



□ 



Proposition 8.6. 



a X k — w n ,fc I (s fc xs„_fc)x(s fc xs„_fc) 



Proof. Very similar to the proof of Prop. 17.91 □ 

Recall from |Saj the definition of cL, - the Littlewood-Richardson coefficients 
defined by the following formula: 

(s p ®snfs:xs n - h =®c x P »s\ 

Ahn 

where p h k and v h n — k. Using the Frobenius reciprocity formula we have for 
every A h n: 
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Claim 8.7. 

p\-k,is hn— k 
phfc,f hn — k 

□ 

We use now the Frobenius reciprocity to obtain the multiplicity of any irreducible 
representation of S n x S n in a X k . 



Proposition 8.8. Let < k < n and A,/ihn. 
m((A,/i),a x *) = 



(XA lf^xS„_ fc >Xp iI^xS„_ fc ) - 



"pv^pv 
p\-k : v\-n — k 



Proof. 

m ((A, /i) , a X h) = (x axk , X(a,a»)) = 

= (XQ„,k ^(S k xS n - k )x(S k xS n - k )'X(\,p)) = 

_ / I Sr,XS„ \ _ 

- U£„,*,)X(A,M) ^(S fc xS„_ fc )x(S , fc xS„_ fc )/ _ 

XCo n . k {^k^n-k,<y k <7 n -k)X\{' K k^n-k)X h i{o'k<y n -k) = 

E Xii„ >fc (TTfcTTn-fc, 7rfc7Tri-fc)XA(7'"fe7rri-fc)XM( 7r fc7rn-fc) = 



1 


(fe!) 2 ((n- 


fc)!) 2 


1 




(fc!) 2 ((n- 


fc)!) 2 


1 




(fc!) 2 ((n- 


fc)!) 2 



= fc!(n-fc)! E XA(7r fe 7r n _ fc )x M (7r fe 7r n _ fc ) = 

n k n„- k eS k xS„_ fc 

= (XA 4|rxS„_ fc 'XM i<s£x,S n _ fc ) = E C pV C pv 

The last equality follows from Claim 18*771 □ 

By the definition of X\ we have a^„ = (Bfc = o a x fc an d thus: 
Corollary 8.9. 

n 

m((\,n),a Bn ) =Y1 E c pV c pV 

□ 

There is a natural mapping between the sets 11% and X\ defined by: 
Hi 9 TrU n , k a ^ nU n ,kcr £ X% 
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One can verify that T is well defined. Moreover, T commutes with the action a of 
S n x S n on X*, i.e.: 

f (ttAct) = nf(A)(j for any A E X%. 

It is easy to see that T is also surjective and thus it induces epimorphisms of 
modules from the S n x S„-module a H k to the S n x S n - module a X k and from the 
Sn-module (3 H u to the S n - module x u ■ Note also that for k = this mapping is 
the identity mapping since ff„ = X® = S n and for k = 1 this mapping is bijective. 
We conclude: 

Claim 8.10. 

m ((A, (i) , a H k) > m ((A, n) , a X k) 

□ 

This implies that if 

V c A & ^ 

p\-h,u\-n— k 

then |A A /x| < 2fc. This can also be seen by the combinatorial interpretation of the 
Littlewood- Richardson coefficients. 

8.2. The action (3 on colored permutations. Recall that every matrix B <= £>„ 
can be written uniquely in the form B = Ztt for some tt € S n and some Z G 
C^ 1 . There exists a natural epimorphism from B n onto defined by omitting the 
minuses: 

p : B n — ► S n p(Ztt) = tt. 

If we restrict p to X^ we obtain a surjective mapping from X\ onto S n which 
commutes with the action a of S n x S n on X„ (and clearly also commutes with 
the action fj of 5 n on by conjugation). It gives us a surjective homomorphism 
from the representation /3 X k onto the conjugacy representation representation of 
S n denoted by ip. Therefore, using the result of |F], we have 

m (A, /3x fc ) > m (\ ip) > f° r an y ^ I - n i 

n 

m (A, /3b,J = m (A, /3x fc ) > for any A h n. 

k=Q 

Although the calculation of xp X k ^ s rather involved the asymptotic results of [R] 
and [AF] can be generalized for the representations (3 X k and 0B n - We start by 
presenting the generalization of Theorem Rl: 

Proposition 8.11. For any < e < 1 there exist < 6(e) and N(e) such that, for 
any partition A of n > N(e) with max{^- , < 6(e), 

m(X,(3 X k) 
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Proof. Let ir G S n - We have to estimate xp xk M = {B G | £?7r = 7i\B}. If 
B € X* commutes with 7r then p(-B) commutes with p(7r) = 7r and thus 

\{B G X„ fe | ttB = Bn}\ < \{B G X„ fc | p(B) G CJ| = (f) |C7 T | . 



By the same considerations we get: 

Xfe„ W = # {B G B n | ttB = Btt} < 2 n |C 7r | . 
In |H] it is shown that for any < e < 1 there exist < (5(e) and iV(e) such 
that, for any partition A of n > N(e) with max{^- , < 5(e), 

MX) - / A l = I E *a(C9 - / A H I E ^a(C)| < £/ A , 

which immediately implies Theorem Rl. (Here m(A) denotes the multiplicity of 
the irreducible representation A in the conjugacy representation of S n .) In our case 
we have xp xk ( e ) = l-^nl = (!■)"' an( ^ therefore 



m(A,/? x *)-("V| = |A E XxWxp x >(*)\< 



which establishes the first statement. Substituting 2 n instead of (?) in this calcu- 
lation we get the second statement. □ 



The generalization of Theorem |AF| is as follows: 
Proposition 8.12. In the notations of Theorem |AF| 



™^°° IIX/3 x J| »^°° ||X/3bJI 

li m ( ff'^' .Bm <ff-*^> -1, 

where k is bounded or tends to infinity remaining less than n. 
Proof. Recall that 

IIX^II 2 = (X { n\x { n } ) = n\ , (x^X/^) = X^(e) = Qn! 

and 04" ] , Xfc„ > = X[i Bn (e) = 2™n! . 
Using the inequality X/3 xfc l 71 ") < (fejl^rl OI Proposition 18 . 1 II we can write 
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Also wc obviously have 

2 



Taking the two last inequalities together we have: 



(n)| 



Also 

(>! < (x^X^J _ (>! < On! ^ 



lll-vo II ~ n \iM 



Substituting 2™ instead of (?) we get similar inequalities for ||X/3s II- ^ n |AF| it is 
proved that 

lim —— = 1 

n-OO ^ C 

ces n 

and thus using the above inequalities the proof is finished. □ 

These asymptotic results can be also obtained for the action (3 (conjugation 
by permutations) on the group C r I S n . Similarly to C B n define the sets 
Y n C C r l S n : 

Definition 8.13. 

Y£ ={ie C r l S n | A has exactly k entries ^ , 1}. 

Note that the sets form a partition of C r I S n and Y% = n!(")(r - l) fc . The 
sets are closed under the action a of S n x S n but they are not transitive under 
this action. 

Consider C™ as the group of diagonal matrices with the entries of the form u> 
(where u> = exp — - the primitive r-th root of unity and < £ < r) on the 
diagonal. Then each matrix A € C r I S n can be uniquely written as A = Za 
for some a E S n and some Z £ C™. Just as in the case of B n , we consider the 
cpimorphism p : B n — > S n defined by: p(Za) = a. 

p induces an cpimorphism of modules between /3yh and the conjugacy represen- 
tation of S n - 

We conclude, using the result of [F|: 

m (A, /9yfc) > to (A, ip) > for any A h n, 
to (A, (3c r is n ) = to (A, /3y n fc) > for any A h n. 

fe=0 

The Theorems Rl and AF are obtained in a way similar to the one we used for 
B n - 

Proposition 8.14. In the conditions and notations of Theorem Rl 

m(A,/3 y *) 
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l- e < ro ^;' g -) <i + e . 



In the notations of Theorem AF 



lim upir-in^w = lim Ugxgj 



\\X/3 Y J n ^°° WXfic 

» 



lim = lim (X%\x0c rl sj = 1 

^Hx^lHlxM ™ llx£V llx/wJI 



where k is bounded or tends to infinity remaining less than n. 

9. Appendix 

We defined the sets H% in order to study the representations (3 H k and a H k. 
These representations can be obtained from the action (3 of S n and the action a of 
S n x S n on some other sets of matrices instead of H%: 

1) In the definition of the matrix U n ,k change only one thing: put the upper left 
k x (n — k) block to be a zero matrix instead of ones in U n ,k- Taking the orbit of 
this matrix under the action (1) of S n x S n we have a natural bijection between 
the obtained set (this orbit) and which preserves the action (1). So we get the 
representations which are isomorphic to the representations (3-^k and a^h . 

2) Instead of wc can take the following subset of Ck+i I S n : 

{A e Ck+i I S n | A has u, uj 2 , . . . , u k exactly once 

and all the other is as in a permutation matix. j 

The representations a and (3 obtained from the corresponding actions on these sets 
are also isomorphic to a H k and P^k . 
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